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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION, 
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(ACQUISITION) 


SUBJECT :  Report  on  Contracting  Practices  at  Air  Force  Laboratory  Facilities 
(Report  No.  D-2007-130) 


We  are  providing  this  report  for  review  and  comment.  We  received  comments  on 
a  draft  of  this  report  from  the  Assistant  Secretary  of  the  Air  Force  (Acquisition).  The 
Assistant  Secretary’s  comments  on  the  draft  of  this  report  conformed  to  the  requirements 
of  DoD  Directive  7650.3.  We  considered  the  Assistant  Secretary’s  comments  on  the  draft 
of  this  report  in  preparing  the  final  report. 

As  a  result  of  the  Assistant  Secretary’s  management  comments,  we  revised  and 
redirected  report  Recommendation  B.  We  request  that  the  Assistant  Secretary  of  the 
Air  Force  (Acquisition)  comment  on  the  revised  R ecomm endati on  B.  by  October  29, 
2007. 


We  request  that  management  provide  comments  that  conform  to  the  requirements 
of  DoD  Directive  7650.3.  If  possible,  please  send  management  comments  in  electronic 
format  (Adobe  Acrobat  file  only)  to  AUDACM@dodig.mil.  Copies  of  the  management 
comments  must  contain  the  actual  signature  of  the  authorizing  official.  We  cannot  accept 
the  /Signed/  symbol  in  place  of  the  actual  signature.  If  you  arrange  to  send  classified 
comments  electronically,  they  must  be  sent  over  the  SECRET  Internet  Protocol  Router 
Network  (SIPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Questions  should  be  directed 
to  Mr.  Benjamin  A.  Mehlman  at  (703)  604-9291  (DSN  664-9291)  or  Mr.  Michael  E. 
Simpson  at  (703)  604-8972  (DSN  664-8972).  See  Appendix  C  for  the  report  distribution. 
The  team  members  are  listed  inside  the  back  cover. 

By  direction  of  the  Deputy  Inspector  General  for  Auditing: 
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Richard  B.  Jolliffe 
Assistant  Inspector  General 
Acquisition  and  Contract  Management 
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Contracting  Practices  at  Air  Force  Laboratory  Facilities 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  DoD  procurement  and  contracting 
personnel  involved  with  contracting  at  DoD  laboratory  facilities  should  read  this  report. 
This  report  discusses  the  need  for  contracting  personnel  to  improve  the  award, 
administration,  and  monitoring  of  contracts  at  Air  Force  laboratories. 

Background.  DoD  laboratories  are  operated  and  managed  by  the  Military  Departments 
to  conduct  research  and  development  and  to  support  acquisition.  One  of  the  main 
functions  performed  by  the  DoD  laboratories  is  science  and  technology  research. 

Science  and  technology  research  includes  basic  research,  applied  research,  and  advanced 
technology  development.  Basic  and  applied  research  are  broadly  stated  areas  of 
scientific  study.  Advanced  technology  development  includes  concept  and  technology 
demonstrations  of  components  and  subsystems  or  system  models.  In  FY  2006  the  DoD 
budget  for  science  and  technology  programs  totaled  $13.3  billion. 

The  Air  Force  Research  Laboratory  is  responsible  for  planning  and  executing  the 
Air  Force  science  and  technology  program  budget  ($2.4  billion  in  FY  2006)  including 
basic  research,  applied  research,  and  advanced  technology  development.  The  Air  Force 
Research  Laboratory  was  formed  in  1997  through  the  consolidation  of  four  former 
Air  Force  laboratories  and  the  Air  Force  Office  of  Scientific  Research.  This  audit 
reviewed  the  management  of  research  and  development  contracts  at  Air  Force 
laboratories.  We  reviewed  20  contracts  valued  in  excess  of  $828  million. 

Results.  The  Air  Force  Research  Laboratory  did  not  place  adequate  emphasis  on 
monitoring  contracts  at  the  research  labs  and  did  not  develop  a  quality  assurance 
surveillance  plan  or  appoint  a  contracting  officer’s  technical  representative  to  perform 
surveillance  on  any  of  the  20  research  and  development  contracts  we  reviewed.  While 
contractors  generally  submitted  project,  technical,  and  fund  status  reports  required  by 
contract,  there  was  no  assurance  that  Government  personnel  were  delegated  the  authority 
and  responsibility  to  analyze  and  act  on  the  contractor  reports.  The  Air  Force  Research 
Laboratory  had  a  material  internal  control  weakness  regarding  oversight  and  surveillance 
of  the  20  research  and  development  contracts  reviewed  during  this  audit.  As  a  result,  the 
laboratories  put  the  Government  at  risk  of  spending  more  than  necessary.  The  Air  Force 
should  establish  guidance  that  each  Air  Force  Research  Laboratory  develop  a  quality 
assurance  surveillance  plan  for  each  contract  to  reduce  the  risk  of  the  Government  paying 
more  for  services  than  the  value  received  and  to  ensure  surveillance  responsibilities  are 
being  carried  out.  In  addition,  the  guidance  should  clearly  define  the  roles  and 
responsibilities  of  contracting  personnel  to  include  appointing  a  trained  contracting 
officer’s  technical  representative  to  monitor  laboratory  contracts.  (See  finding  A  for  the 
detailed  recommendations.) 


The  Air  Force  Research  Laboratory  had  established  adequate  internal  controls  over  the 
research  and  development  contract  award  process,  including  compliance  with  applicable 
sections  of  the  Federal  Acquisition  Regulation.  Air  Force  Research  Laboratory  contract 
file  documents  for  18  of  20  contracts  we  reviewed  contained  an  overview  of  the  source 
selection  identifying  the  original  bidders  and  explaining  how  the  winner  was  selected, 
even  though  the  Air  Force  Research  Laboratory  did  not  have  a  contracting  policy  to 
require  documentation  of  source  selection  reasoning.  Without  source  selection 
documentation,  an  audit  trail  of  the  major  procuring  events  is  unavailable,  resulting  in  a 
loss  of  the  history  of  business  judgments  and  trade-offs  made  by  the  Government.  The 
Air  Force  should  revise  its  Instructions  to  require  that  price  negotiation  memorandums 
explain  the  rationale  for  selecting  the  winning  proposal  and  business  judgments  and 
trade-offs  made  in  connection  with  the  selection  in  accordance  with  Federal  Acquisition 
Regulation  requirements.  (See  finding  B  for  the  detailed  recommendations). 

Management  Comments  and  Audit  Response.  The  Assistant  Secretary  of  the 
Air  Force  (Acquisition)  concurred  with  the  intent  of  the  findings  and  the  recommendation 
to  issue  guidance  regarding  Air  Force  Research  Laboratory  quality  assurance  surveillance 
and  to  define  the  roles  and  responsibilities  of  contracting  personnel.  The  Assistant 
Secretary  noted  that  the  primary  Air  Force  Research  Laboratory  quality  assurance 
guidance  will  be  replaced  in  favor  of  a  revised  Air  Force  Instruction  63-101,  “Acquisition 
and  Sustainment  Lifecycle  Management,”  by  December  2007.  The  Assistant  Secretary 
nonconcurred  with  a  draft  report  recommendation  made  to  the  Commander,  Air  Force 
Research  Laboratory  to  revise  internal  laboratory  guidance.  As  a  result  of  the  Assistant 
Secretary’s  comments,  we  revised  draft  report  Recommendation  B.  in  the  final  report  and 
redirected  the  recommendation  to  the  Assistant  Secretary.  We  request  that  the  Assistant 
Secretary  comment  on  the  revised  recommendation  by  October  29,  2007.  See  the 
Findings  section  for  a  discussion  of  management  comments  and  the  Management 
Comments  section  of  the  report  for  the  complete  text  of  the  comments. 
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Background 


Department  of  Defense  laboratories  are  operated  and  managed  by  the  Military 
Departments  to  conduct  research  and  development  and  support  acquisition.  The 
purpose  of  a  research  and  development  program  is  to  advance  scientific  and  technical 
knowledge  and  apply  that  knowledge  to  the  extent  necessary  to  achieve  agency  and 
national  goals.  Research  and  development  contracts  are  considered  performance- 
based  cost  reimbursement  service  contracts  and  should  include  a  quality  assurance 
surveillance  plan  (QASP)  that  describes  how  the  contractor’s  performance  will  be 
measured  against  its  quality,  quantity,  and  timeliness.  DoD  laboratories  include  the 
Army  laboratories,  the  Navy  laboratories,  and  the  Air  Force  laboratories,  and  they 
conduct  science  and  technology  research.  DoD  science  and  technology  research 
includes  basic  research,  applied  research,  and  advanced  technology  development. 
Basic  research  is  systematic  study  toward  greater  knowledge  without  a  specific 
application  in  mind.  Applied  research  is  also  systematic  study  to  understand  the 
means  to  meet  a  recognized  and  specific  need.  Advanced  technology  development 
includes  concept  and  technology  demonstrations  of  components  and  subsystems  or 
system  models.  The  DoD  science  and  technology  program  was  budgeted  at 
approximately  $13.3  billion  for  FY  2006. 

This  audit  was  initiated  from  findings  in  DoD  Inspector  General  Report 
No.  D-2007-036,  “Contracting  Practices  at  the  Major  Range  and  Test  Facilities  Base,” 
December  27,  2006,  that  contracting  officials  did  not  adequately  manage  10  contracts 
at  the  DoD  Major  Range  and  Test  Facilities  Base  locations. 1 

The  Air  Force  Research  Laboratory  (AFRL)  has  overseen  years  of  critical  research 
efforts  for  the  Air  Force  and  DoD.  Congress  formalized  DoD  support  for  basic 
research  by  establishing  the  Air  Force  Office  of  Scientific  Research  in  1952.  AFRL  is 
headquartered  at  Wright-Patterson  Air  Force  Base,  Ohio,  and  includes  facilities 
consolidated  from  four  former  Air  Force  laboratories  and  the  Office  of  Scientific 
Research  in  1997.  AFRL  is  responsible  for  planning  and  executing  the  Air  Force 
science  and  technology  program  including  basic  research,  applied  research,  and 
advanced  technology  development.  The  Air  Force  science  and  technology  program 
was  budgeted  at  approximately  $2.4  billion  for  FY  2006. 

AFRL’s  mission  is  to  discover,  develop,  integrate,  and  deliver  affordable  technologies 
for  improved  warfighting  capabilities  for  America’s  aerospace  forces.  AFRL 
accomplishes  its  mission  through  nine  technology  directorates  located  throughout  the 
United  States,  and  the  Air  Force  Office  of  Scientific  Research.  We  reviewed 
10  contracts  managed  by  contract  officials  at  Wright-Patterson  Air  Force  Base  and 
10  contracts  managed  by  contract  officials  located  at  Kirtland  Air  Force  Base,  New 
Mexico. 


1  The  10  Major  Range  and  Test  Facilities  Base  contracts  reviewed  did  not  have  adequate  contract  administration 
or  surveillance,  and  contractors  had  no  incentive  to  control  costs. 
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Objective 


Our  overall  audit  objective  was  to  review  the  management  of  contracts  at  DoD 
laboratories.  This  report  reviewed  the  management  of  research  and  development 
contracts  at  Air  Force  laboratories.  Specifically,  the  review  consisted  of  the  adequacy 
of  the  contract  award  process  and  oversight  of  contracts  and  contractor  personnel  to 
determine  whether  the  contracts  comply  with  applicable  sections  of  the  Federal 
Acquisition  Regulation  (FAR).  See  Appendix  A  for  a  discussion  of  the  scope  and 
methodology  and  for  prior  coverage  related  to  the  objective. 


Review  of  Internal  Controls 


We  identified  material  internal  control  weaknesses  for  AFRL  as  defined  by  DoD 
Instruction  5010.40,  “Managers’  Internal  Control  (MIC)  Program  Procedures,” 

January  4,  2006.  AFRL  controls  for  contract  surveillance  were  not  adequate  to  ensure 
that  surveillance  or  monitoring  of  contracts  was  maintained.  AFRL  policy  does  not 
require  that  contract  officials  develop  a  QASP  or  appoint  a  contracting  officer 
technical  representative  (COTR)  to  conduct  surveillance  on  any  of  the  contracts  we 
reviewed.  Recommendation  A.,  if  implemented,  will  improve  AFRL  contract 
surveillance  procedures  with  additional  oversight  to  reduce  the  risk  of  the  Government 
paying  more  than  the  value  of  services  incurred.  A  copy  of  the  report  will  be  sent  to 
the  senior  official  responsible  for  the  internal  controls  in  AFRL. 
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A.  Contract  Surveillance  at  Air  Force 
Research  Laboratory  Facilities 

AFRL  did  not  place  adequate  emphasis  on  monitoring  contracts  at  the  research 
labs  and  did  not  develop  QASPs  for  any  of  the  20  research  and  development 
contracts  we  reviewed.  In  addition,  AFRL  contracting  officers  did  not  appoint 
COTRs  for  any  of  the  20  contracts.  This  occurred  because  Air  Force  policy  for 
surveillance  is  ambiguous  and  is  based  on  past  AFRL  practices.  Although 
contractors  generally  submit  project,  technical,  and  fund  status  reports,  this  is 
not  an  adequate  substitute  for  Air  Force  surveillance.  As  a  result,  the  AFRL 
put  the  Government  at  risk  of  increased  research  and  development  costs  and 
technical  risk. 


Government  Contract  Surveillance  Guidance 


Quality  Assurance  Surveillance  Requirements.  FAR  Subpart  4.8,  “Government 
Contract  Files,”  provides  requirements  for  establishing,  maintaining,  and  disposing  of 
contract  files.  FAR  4.803  (b),  “Contract  Administration  Office,”  states  that  production 
surveillance  records  and  quality  assurance  records  should  be  a  part  of  the  contract  file. 
FAR  Part  16,  “Types  of  Contracts,”  discusses  the  types  of  contracts  that  may  be  used 
in  acquisition.  FAR  16.301 -3(a)(2),  “Limitations,”  states  that  cost  reimbursement 
contracts  may  be  used  only  when  appropriate  Government  surveillance  during 
performance  will  provide  reasonable  assurance  that  efficient  methods  and  effective 
cost  controls  are  used.  FAR  46. 103,  “Contracting  Officer  Responsibilities,”  states  that 
contracting  offices  are  responsible  for  receiving  technical  requirements  and  any 
specifications  for  inspection,  testing,  and  other  contract  quality  essential  to  ensure  the 
integrity  of  the  supplies  or  services.  The  activity  responsible  for  technical 
requirements  also  is  responsible  for  prescribing  contract  quality  requirements,  such  as 
inspection  and  testing  of  service  contracts.  FAR  46.401(a),  “Government  Contract 
Quality  Assurance,”  states  that  a  QASP  should  be  prepared  in  conjunction  with  the 
preparation  of  the  statement  of  work. 

COTR  Requirement.  Defense  Federal  Acquisition  Regulation  Supplement 
Section  201.6,  “Contracting  Authority  and  Responsibilities,”  states  that  contracting 
officers  may  designate  qualified  personnel  as  their  authorized  representatives  to  assist 
in  either  technical  monitoring  or  administration  of  a  contract.  To  assist  in 
administrative  duties,  contracting  officers  are  authorized  to  designate  qualified 
personnel  as  COTRs.2  The  designated  COTRs  must  act  as  the  eyes  and  ears  for  the 
contracting  officer.  COTRs  must  be  properly  trained,  designated  in  writing,  and 
maintain  contract  surveillance  files.  Designation  letters  signed  by  the  contracting 
officer  should  specify  the  extent  and  limitations  of  the  COTR  authority  to  act  on 
behalf  of  the  contracting  officer. 

The  primary  role  of  a  COTR  is  to  provide  technical  clarification  and  to  monitor 
contract  performance  closely  to  ensure  the  Government  pays  only  for  the  services  and 
materials  under  the  contract.  To  avoid  misunderstandings,  disagreements,  unnecessary 


2  COTRs  are  also  referred  to  as  contracting  officer  representatives. 
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costs,  and  to  maintain  proper  control  of  the  contract  and  maintain  adequate  file 
documentation,  the  Government  should  ensure  that  all  COTR  technical  directions  are 
in  writing. 

AFRL  Internal  Contract  Policy.  Air  Force  Instruction  63-124,  “Performance-Based 
Services  Acquisition,”  August  1,  2005,  section  1.4.5,  states  that  “performance  metrics 
are  to  be  used  to  track  contractor  progress  towards  meeting  stated  performance 
objectives.  The  multi-functional  team  in  assessing  contractor  perfonnance  validates 
that  the  perfonnance  metrics  align  with  the  performance-based  work  statement  and 
overall  mission  supports  objectives.”  The  AFRL  applies  principles  from  Air  Force 
Instruction  63-124  to  its  research  and  development  Management  and  Oversight  of 
Acquisition  of  Services  Processes  (MOASP)  guidance. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2002,  section  2330, 
“Procurement  of  services;  management  structure,”  December  2001,  discusses 
contracting  responsibilities  of  designated  officials  to  ensure  the  services  procured  are 
in  the  best  interest  of  DoD  and  are  managed  in  compliance  with  applicable  statutes, 
regulations,  directives,  and  other  requirements. 


AFRL  Contract  Surveillance 


In  the  20  contracts  reviewed  that  were  valued  in  excess  of  $828  million,  AFRL  used  a 
generic  policy  for  contract  surveillance  that  was  based  on  past  AFRL  practices.  AFRL 
did  not  tailor  each  contract  to  the  specific  needs  of  each  research  and  development 
service  contract. 

AFRL  Contract  Surveillance  Process.  The  Air  Force  Materiel  Command  (AFMC) 
February  11,  2004,  MOASP  guidance  states  that  research  and  development  contract 
oversight  is  accomplished  via  contractor  periodic  reporting  requirements  included  in  a 
Contract  Data  Requirements  List.  However,  the  MOASP  guidance  is  a  generic 
document  and  does  not  specify  how  surveillance  tasks  are  to  be  completed.  For 
example,  the  MOASP  guidance  does  not  indicate  how  the  data  in  the  Contract  Data 
Requirements  List,  such  as  funds/man-hour  reports,  final  reports,  and  status  reports, 
would  be  assessed  by  the  program  manager  or  technical  manager.  It  also  doesn’t 
address  surveillance  that  would  be  necessary  beyond  reviewing  and  analyzing  reports 
provided  by  the  contractor. 

The  MOASP  guidance  states  that  AFRL  Project  Management  Reviews,  Technical 
Management  Reviews,  and  Govemment/contractor  interaction  support  research  and 
development  contract  oversight.  But  the  MOASP  guidance  provides  no  requirements 
for  the  perfonnance  of  the  management  reviews.  The  MOASP  guidance  lists 
documents  that  could  be  gathered  to  perfonn  a  surveillance  review,  but  makes  no 
mention  of  the  process  used  to  validate  data  or  support  conforming  or  nonconforming 
observations.  The  MOASP  guidance  also  does  not  indicate  the  types  or  frequency  of 
reviews  of  contractor-incurred  cost.  Without  the  detail  as  required  by  a  QASP 
prepared  by  the  requiring  activity,  neither  contracting  nor  oversight  officials  can 
determine  whether  contract  monitoring  efforts  are  sufficient. 

Air  Force  officials  noted  that  other  AFRL  internal  guidance  required  a  laboratory 
management  review  process  to  rapidly  identify  potential  problems  and  assure  that 
appropriate  management  level  have  accurate,  timely,  and  pertinent  information. 
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Air  Force  officials  also  stated  that  the  other  AFRL  guidance  requires  semi-annual 
baseline  reviews  on  all  research  and  development  contracts  covering  program  status 
over  manning,  cost,  schedule,  technical  performance  and  deliverables,  funding,  and 
contracting.  Air  Force  officials  stated  that  the  laboratory  management  reviews  and 
baseline  reviews  are  used  in  lieu  of  a  QASP  process. 

The  AFRL  MOASP  guidance  does  not  instruct  contracting  officials  to  prepare  a  QASP 
as  required  by  FAR  46.401(a).  We  concluded  that  the  MOASP  guidance  was 
inadequate  to  perform  AFRL  surveillance  efforts.  An  adequate  surveillance  plan 
provides  the  foundation  for  a  comprehensive  and  systematic  monitoring  of  contractor 
performance  and  a  standard  against  which  actual  surveillance  efforts  can  be  measured. 
The  lack  of  a  surveillance  plan  subjects  the  Government  to  greater  risk  that  the 
contractor  may  not  be  performing  all  contractual  requirements  in  accordance  with  the 
contract  tenns. 

AFRL  COTRs.  While  AFRL  contracting  officials  verbally  assigned  technical 
representatives  to  monitor  and  provide  oversight  of  cost-type  research  and 
development  contracts,  there  was  no  letter  of  designation  of  the  COTR  appointment  as 
required  by  Defense  Federal  Acquisition  Regulation  Supplement  Section  201.6  in  the 
contract  files.  Contracting  officials  at  AFRL  appointed  a  multifunctional  team  to 
manage  and  provide  oversight  for  each  contract.  However,  the  duties  and 
responsibilities  of  the  multifunctional  team  were  not  clearly  stated.  AFRL  did  not 
appoint  COTRs  because  it  was  not  addressed  by  the  AFRL  MOASP  guidance. 

AFRL  Surveillance  Plans.  AFRL  officials  did  not  develop  a  QASP  for  research  and 
development  contracts  and  do  not  require  a  surveillance  plan  to  be  a  part  of  the 
contracting  file.  AFRL  contracting  officials  stated  that  the  MOASP  guidance 
substituted  for  the  QASP  and  assured  adequate  contractor  reporting,  oversight,  and 
monitoring  of  progress  and  performance.  AFRL  contracting  officials  stated  that  the 
QASP  would  duplicate  reporting  and  oversight  requirements  already  included  by 
contract  or  inherent  in  AFRL  operations. 

We  do  not  believe  that  the  MOASP  guidance  meets  monitoring  of  contractor 
performance  as  required  by  FAR  16.301 -3(a)  (2)  and  FAR  46.401(a).  AFRL  should 
prepare  QASPs  in  conjunction  with  the  statement  of  work  and  indicate  which  work 
requires  surveillance  and  which  type  of  surveillance  will  be  performed. 


AFRL  Oversight  of  Acquisition  of  Services  Policy 


AFRL  Policy.  The  National  Defense  Authorization  Act  for  Fiscal  Year  2002 
(Section  801  of  Public  Law  107-107)  directs  each  Military  Service  to  establish  a 
designated  official  and  management  structure  for  services  contracts,  and  required  DoD 
to  establish  sound  management  and  oversight  processes  for  service  contracts.  On 
June  3,  2003,  the  Assistant  Secretary  of  the  Air  Force  (Acquisition)  issued  a 
memorandum  to  AFMC  to  implement  the  Section  801  requirement  for  services 
acquisitions  at  each  Air  Force  major  command  through  a  formalized  service  contract 
management  and  oversight  process.  On  March  8,  2004,  the  Air  Force  Materiel 
Command  Commander  issued  a  memorandum  to  the  Air  Force  Materiel  Center 
Commanders  noting  that  the  MOASP  guidance  would  be  effective  as  policy  for 
Section  801  implementation  for  research  and  development  service  contracts.  On 
May  13,  2004,  the  Air  Force  Program  Executive  Officer  for  Combat  and  Mission 
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Support  issued  a  memorandum  that  gave  AFRL  Center  Commanders  authority  to  act 
as  the  designated  official  for  management  and  oversight  of  all  services  acquisitions 
within  the  AFRL.  The  memorandum  also  stated  AFRL  would  work  with  AFMC  to 
amend  the  MOASP  guidance  to  include  AFRL  specific  procedures. 

On  July  19,  2004,  the  AFRL  Commander  issued  a  memorandum  to  all  AFRL  activities 
stating  that  AFRL,  AFMC,  and  Air  Force  Program  Executive  Officer  Combat  and 
Mission  Support  had  jointly  determined  that  Air  Force  research  and  development 
contracting  fell  under  the  purview  of  the  Section  801  definition  of  services. 

Section  801  requires  that  acquisitions  of  services  within  DoD  are  based  on  clear, 
performance-based  requirements  and  that  required  outcomes  are  identified  and 
measurable,  and  that  acquisitions  are  properly  planned  and  administered  to  achieve  the 
intended  results.  The  AFRL  memorandum  also  stated  that  the  MOASP  guidance  was 
the  AFRL  policy  for  Section  801  pre-  and  post-award  management  and  oversight  of 
research  and  development  contracts.  We  believe  that  the  Air  Force  use  of  the  MOASP 
guidance  to  implement  Section  801  for  Air  Force  research  and  development 
contracting  effectively  side-stepped  the  intent  of  Section  801  by  grandfathering 
existing  AFRL  oversight  practices  that  are  not  based  on  performance-based 
requirements  and  outcomes. 


Conclusion 


AFRL  contracting  officials  did  not  prepare  a  QASP  on  any  of  the  20  contracts 
reviewed.  To  provide  reasonable  assurance  of  quality  performance  on  service 
contracts  AFRL  should  create  a  QASP  while  developing  the  statement  of  work.  To 
ensure  the  Government  receives  good  value,  specific  attention  is  required  on  cost 
reimbursement  contracts.  In  addition,  contracting  officials  did  not  appoint  a  COTR. 
Failure  to  appoint  a  COTR  on  research  and  development  service  contracts  subjects  the 
Government  to  greater  risk  of  unnecessary  costs  and  inadequate  technical 
administration  on  contract  performance.  Also,  because  of  the  technical  complexity 
specifically  on  research  and  development  service  contracts,  contracting  officials 
should  prepare  a  designation  letter  tailored  to  the  needs  of  each  specific  contract 
addressing  the  responsibilities  of  the  requirement.  Oversight  and  monitoring  of 
contractor  performance  is  vital  in  protecting  the  interest  of  the  Government. 

The  FAR  authorizes  the  use  of  cost  reimbursement  contracts  only  when  appropriate 
Government  surveillance  during  performance  will  provide  reasonable  assurance  that 
efficient  methods  and  effective  cost  controls  are  used.  AFRL  contracting  officials 
should  update  the  MOASP  guidance  to  require  each  AFRL  directorate  to  develop  a 
QASP  for  each  contract  and  appoint  in  writing  trained  contracting  officials  to  ensure 
surveillance  duties  and  responsibilities  are  clearly  defined  on  service  contracts. 


Management  Comments  on  the  Finding  and  Audit  Response 


Air  Force  Comments.  The  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
concurred  with  the  finding,  noting  that  the  Air  Force  has  increased  emphasis  on 
defining  requirements  to  ensure  surveillance  duties  are  adequately  performed.  The 
Assistant  Secretary  noted  that  AFRL  used  the  MOASP  to  ensure  adequate  surveillance 
performance.  The  Assistant  Secretary  agreed  that  Air  Force  Instruction  63-124  directs 
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performance-based  procedures  for  developing  requirements,  but  also  allows 
acquisition  strategies  (such  as  the  MOASP)  to  deviate  from  use  of  a  Performance  Plan 
or  QASP  as  long  as  the  deviations  are  documented  in  the  contract  file.  Such  MOASP 
deviation  documentation  applied  to  all  AFRL  research  and  development  contracts  and 
task  orders.  The  Assistant  Secretary  of  the  Air  Force  (Acquisition)  concurred  with  the 
finding,  noting  that  the  Air  Force  has  increased  emphasis  on  defining  requirements  to 
ensure  surveillance  duties  are  adequately  performed. 

The  Assistant  Secretary  noted  that  AFRL  manages  contracted  and  in-house  scientific 
research  and  development  in  accordance  with  Air  Force  Policy  Directive  61-1, 
“Management  of  Science  and  Technology”  and  AFRL  Instruction  61-202,  “AFRL 
Laboratory  Management  Review  (LMR)  Process,”  and  other  AFRL  guidance  to  assure 
management  has  appropriate,  timely,  and  pertinent  information  through  baseline 
reviews  and  reports  on  contractor  cost,  schedule,  and  technical  milestones.  Program 
managers  or  project  engineers  review  the  baseline  reviews  with  contractors.  The 
Assistant  Secretary  noted  that  such  reviews  are  used  by  AFRL  in  lieu  of  a  QASP  or  a 
performance  plan.  The  Assistant  Secretary  also  noted  that  AFRL  program  managers 
and  project  engineers  are  the  “ad  hoc  appointed  COTRs”  on  AFRL  research  and 
development  contracts. 

Audit  Response.  Based  on  the  Air  Force  comments,  we  expanded  discussion  in  the 
final  report  regarding  AFRL  internal  guidance  and  the  AFRL  contract  surveillance 
process.  However,  we  continue  to  conclude  that  the  AFRL  surveillance  process  falls 
short  of  the  detail  as  required  by  a  QASP  process  and  that  AFRL  guidance  does  not 
meet  the  standard  to  monitor  contractor  performance  as  required  by 
FAR  16.301  3(a)  (2)  and  FAR  46.401(a).  The  lack  of  a  surveillance  plan  subjects  the 
Government  to  greater  risk  that  the  contractor  may  not  be  performing  all  contractual 
requirements  in  accordance  with  the  contract  terms.  AFRL  should  prepare  QASPs  in 
conjunction  with  the  statement  of  work  and  indicate  which  work  requires  surveillance 
and  which  type  of  surveillance  will  be  performed.  Use  of  the  program  managers  and 
project  engineers  as  ad  hoc  COTRs  falls  short  of  DoD  requirements  for  contracting 
officers  to  appoint  COTRs  to  conduct  contract  surveillance. 


Recommendation  and  Management  Comments 


A.  We  recommend  that  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition)  require  the  Air  Force  Research  Laboratory  to  revise  the 
Management  and  Oversight  of  Acquisition  Services  Processes  to  include  guidance 
that  each  Air  Force  Research  Laboratory  develop  a  quality  assurance 
surveillance  plan  for  each  contract  to  reduce  the  risk  of  the  Government  paying 
more  for  services  than  the  value  received  and  to  ensure  that  surveillance 
responsibilities  are  being  carrying  out.  In  addition,  the  guidance  should  clearly 
define  the  roles  and  responsibilities  of  contracting  personnel  to  include 
appointing  a  trained  contracting  officer’s  technical  representative  to  monitor 
technical  service  performance  contracts. 

Air  Force  Comments  on  the  Recommendation.  The  Assistant  Secretary  of 
the  Air  Force  (Acquisition)  concurred  with  the  recommendation.  The  Assistant 
Secretary  noted  that  the  AFRL  MOASP  will  be  removed  in  favor  of  a  revised 
Air  Force  Instruction  63-101,  “Acquisition  and  Sustainment  Lifecycle  Management,” 
by  December  2007.  The  Assistant  Secretary  restated  that  AFRL  utilizes  unique 
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oversight  procedures  such  as  program  management  reviews,  contract  data 
requirements,  and  technical  management  reviews  to  ensure  quality.  The  Assistant 
Secretary  also  noted  that  the  uncertain  nature  of  research  and  development  precludes 
the  development  and  application  of  objective  criteria  for  perfonnance  assessment. 
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B.  Air  Force  Research  Laboratories 
Research  and  Development  Contracts 
Source  Selection  Process 

AFRL  established  adequate  internal  controls  over  the  research  and 
development  contract  award  process  and  compliance  with  applicable  sections 
of  the  FAR.  AFRL  contract  file  documents  for  18  of  20  contracts  we  reviewed 
contained  an  overview  of  the  source  selection  identifying  the  original  bidders 
and  explaining  how  the  winner  was  selected,  even  though  AFRL  did  not  have  a 
contracting  policy  to  require  documentation  of  source  selection  reasoning. 
Without  source  selection  documentation,  an  audit  trail  of  the  major  procuring 
events  is  unavailable,  resulting  in  a  loss  of  the  history  of  business  judgments 
and  trade-offs  made  by  the  Government,  including  benefits  associated  with 
additional  costs  to  the  acquisition. 


AFRL  Contract  Award  Procedures 


AFRL  Proposal  Evaluation  Process.  The  AFRL  “Broad  Agency  Announcement 
Industry  Guide,”  May  2006,  directs  that  research  and  development  proposals  be 
evaluated  based  on  the  criteria  published  in  the  announcement  by  a  Government 
technical  evaluation  team  or,  when  the  necessary  expertise  cannot  be  found,  outside 
consultants  may  be  included  in  the  technical  evaluation  team.  AFRL  technical 
evaluations  classify  bidder  proposals  into  three  categories.  Proposals  rated  Category  I 
are  recommended  for  acceptance  and  normally  will  be  displaced  only  by  other 
Category  I  proposals.  Category  II  proposals  are  rated  at  a  lower  priority  than 
Category  I  but  can  be  recommended  for  acceptance,  subject  to  available  funds. 
Category  III  proposals  are  detennined  by  AFRL  to  not  be  technically  sound  or  judged 
not  to  meet  agency  needs. 

AFRL  Award  Process.  Once  AFRL  completes  the  technical  evaluation,  it  negotiates 
the  most  appropriate  cost  and  technical  arrangement  with  the  offerors  selected  for 
award.  The  types  of  awards  include  contracts,  grants,  or  cooperative  agreements. 
AFRL  awards  are  usually  made  to  Category  I  proposals  prior  to  awards  made  to  any 
Category  II  proposals.  Occasionally,  the  Air  Force  may  buy  only  a  certain  portion  of  a 
proposal  depending  on  available  funding  from  AFRL  divisions. 


Criteria 


FAR  criteria  addresses  source  selection  decisions  in  several  different  sections. 

FAR  4.8,  “Government  Contract  Files,”  prescribes  the  requirements  for  establishing 
and  maintaining  the  contract  files.  The  guidance  directs  that  the  documentation  should 
constitute  a  complete  history  of  the  transaction  and  a  complete  background  as  a  basis 
for  supporting  infonned  decisions  at  each  step  of  the  acquisition  process.  FAR  4.803, 
“Contents  of  Contract  Files,”  lists  examples  of  records  in  the  contracting  office  files 
and  includes  source  selection  documentation  as  one  of  the  items.  FAR  15.308, 

“Source  Selection  Decision,”  states  that  the  source  selection  decision  shah  be 
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documented  and  the  documentation  shall  include  the  rationale  for  any  business 
judgments  and  trade-offs  made  regarding  contract  cost,  economy,  and  efficiency. 

FAR  Part  35,  “Research  and  Development  Contracting,”  describes  policies  and 
procedures  specific  to  research  and  development  contracting.  Specifically, 

FAR  35.016,  “Broad  Agency  Announcement,”  describes  procedures  for  the 
acquisition  of  basic  and  applied  research  not  related  to  the  development  of  a  specific 
system,  and  where  the  agency  anticipates  proposals  with  varying  technical  or  scientific 
approaches.  Air  Force  Materiel  Command  FAR  Supplement  Part  5335,  “Research  and 
Development  Contracting,”  describes  the  receipt  and  evaluation  of  proposals  into 
Category  I,  II,  or  III.  The  National  Defense  Authorization  Act  for  Fiscal  Year  2002 
(Public  Law  107-107)  requires  the  Secretary  of  Defense  to  establish  a  management 
structure  for  the  procurement  of  services  for  DoD.  It  was  determined  within  the 
Air  Force  that  research  and  development  falls  under  the  category  of  services.  In  May 
2004,  the  AFRL  MOASP  guidance  established  policy  for  management  and  oversight 
of  research  and  development  services  contracts. 


AFRL  Source  Selection  Decisions 


Results  of  Source  Selection  Information  Analysis.  While  AFRL  established  and 
maintained  the  research  and  development  contract  award  files  that  were  generally 
well-organized  and  complete,  the  AFRL  contract  files  and  negotiation  memorandums 
did  not  always  contain  an  overview  of  the  source  selection  process  identifying  the 
original  bidders  and  explaining  how  the  winner  was  selected.  The  AFRL  source 
selection  process  was  documented  in  two  ways,  via  the  technical  evaluation  and  also 
in  the  price  negotiation  memorandum  (PNM).3  AFRL  technical  evaluations  generally 
contained  detailed  analyses  of  required  elements  of  each  bidder  proposal.  However, 
AFRL  PNMs  only  briefly  summarize  the  source  selection  decisions  resulting  from  the 
technical  evaluations  in  a  sentence  or  two.  In  two  instances,  the  number  and  identity 
of  proposals  and  resulting  source  selection  decisions  were  not  disclosed  in  the  PNM. 

•  For  contract  FA9453-05-C-0220,  valued  at  $  13.4  million,  the  PNM  did  not 
disclose  the  competing  proposals  and  the  basis  for  selecting  the  winning 
proposal  over  the  other  bidders.  However,  the  technical  evaluation  included  a 
detailed  narrative  that  summarized  the  scientific/technical  strengths  and 
weaknesses  of  each  of  four  proposals  including  the  recommended  winner.  The 
technical  evaluation  noted  that  proposed  costs  for  all  bidders  were  reasonable 
but  provided  no  elaboration. 

•  For  contract  modification  F29601-03-C-0061  P00014,  valued  at  $6  million,  the 
PNM  for  the  basic  contract  did  not  identify  the  other  three  proposals  nor 
explain  why  the  winning  proposal  was  selected.  The  technical  evaluation 
explained  that  the  proposed  technical  approach  for  the  winner  was  relatively 
sound  and  that  the  proposal  provided  adequate  cost  information.  But  the 
technical  evaluation  did  not  identify  or  address  any  other  proposal.  This 
contract  was  completed  in  June  2006  and  closed  out  in  August  2006. 


3  AFRL  also  referred  to  the  “price  negotiation  memorandum”  as  the  “price  competition  memorandum.” 
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For  the  two  contracts,  AFRL  PNMs  did  not  provide  the  source  selection  information 
with  respect  to  other  bidder  proposals  nor  discuss  the  basis  on  which  the  winning 
proposal  was  selected. 

AFRL  as  Agent  for  Defense  Advanced  Research  Projects  Agency  Procurements. 

AFRL  acted  as  the  procurement  agent  for  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  for  3  of  the  20  AFRL  contracts  reviewed.  DARPA  solicited, 
funded,  and  performed  all  acquisition  planning  and  source  selection  activities  related 
to  the  three  contracts.  For  contracts  FA8650-04-C-7146  and  FA8650-04-C-7108, 
DARPA  furnished  a  memorandum  to  the  AFRL  contracting  officer  describing  the 
basis  used  to  select  the  winning  proposal,  including  overall  scientific  and  technical 
merit,  relevance  to  the  DARPA  mission,  and  adequate  cost  documentation.  Contract 
FA8650-05-C-7214  included  an  AFRL  technical  evaluation  and  PNM  summarizing 
DARPA  source  selection  determinations  and  AFRL  evaluated  technical  and  cost 
aspects.  AFRL  contract  officers  coordinated  with  DARPA  in  the  contract  award 
process  and  adequately  documented  the  contract  files  to  support  the  contract  award. 


AFRL  Contract  Award  Policy 


AFRL  did  not  establish  or  maintain  contracting  policy  to  require  source  selection  in 
the  PNM  or  contract  file  addressing  unsuccessful  proposals  or  the  rationale  for  the 
selection  decisions.  AFRL  relies  heavily  on  the  MOASP  guidance  and  the  Broad 
Agency  Announcement  Industry  Guide  for  its  contract  management  policy.  The 
MOASP  guidance  is  written  at  a  very  general  level  and  does  not  address  such  specifics 
as  evaluation  for  award. 

AFRL  contracting  officials  stated  their  acquisitions  are  not  formal  source  selections 
and  therefore  are  not  subject  to  the  specific  process  and  documentation  requirements 
in  FAR  Part  15.  We  disagree  in  that  FAR  15.308  states  that  the  source  selection 
decision  shall  be  documented  and  the  documentation  shall  include  the  rationale  for  any 
business  judgments  and  trade-offs  made.  While  FAR  Part  35  discusses  a  revised 
negotiation  process  for  research  and  development  contracts,  it  also  references  FAR 
Part  15  for  negotiated  research  and  development  contract  actions.  AFRL  contracting 
officials  stated  that  the  existing  AFRL  Broad  Agency  Announcement  Industry  Guide 
and  MOASP  technical  evaluation  process  substitutes  for  a  fonnal  source  selection 
decision  document.  We  believe  that  AFRL  should  continue  to  use  the  Broad  Agency 
Announcement  Industry  Guide  and  MOASP  guidance  but  revise  them  to  require 
documentation  of  source  selection  decisions  in  accordance  with  FAR  15.308  and 
FAR  4.803. 


Conclusion 


AFRL  established  and  maintained  research  and  development  contract  award  files  that 
were  generally  well-organized  and  complete.  However,  AFRL  contract  files  and 
PNMs  did  not  always  contain  an  overview  of  the  source  selection  process  identifying 
the  original  bidders  and  explaining  how  the  winner  was  selected.  AFRL  research  and 
development  acquisitions  are  subject  to  FAR  Part  15  regarding  disclosure  of  source 
selection  requirements  and  AFRL  guidance  should  direct  that  PNMs  contain  more 
complete  disclosure  of  the  competing  proposals  and  the  basis  for  selecting  the  winning 
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proposal  over  the  others.  Without  source  selection  documentation,  an  audit  trail  of  the 
major  procurement  events  is  unavailable  resulting  in  a  loss  of  the  history  of  business 
judgments  and  trade-offs  made,  including  the  rationale  when  higher-priced  proposals 
were  selected. 


Management  Comments  on  the  Finding  and  Audit  Response 


Air  Force  Comments.  The  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
concurred  with  the  finding,  agreeing  that  documentation  of  the  source  selection 
process  in  accordance  with  FAR  Part  15,  or  documentation  of  the  peer  or  scientific 
reviews  in  accordance  with  FAR  35.016,  is  required  in  the  contract  file.  The  Assistant 
Secretary  noted  that  AFRL  did  not  provide  additional  policy  on  selection 
documentation  and  price  reasonableness  due  to  existing  Air  Force  and  AFMC 
guidance.  The  Assistant  Secretary  noted  that  evaluation  of  proposals  received  under 
broad  agency  announcements  are  not  necessarily  evaluated  against  each  other  since 
they  are  not  submitted  in  accordance  with  a  common  work  statement.  The  Assistant 
Secretary  agreed  that  contract  F29601-03-C-0061  did  not  contain  adequate 
documentation  but  noted  that  there  was  no  systemic  problem  across  AFRL.  The 
Assistant  Secretary  noted  that  the  need  for  inclusion  of  documentation  would  be 
included  in  training  for  all  contracting  officers  and  negotiators. 

Audit  Response.  Based  on  our  review  of  the  20  research  and  development  contracts, 
we  found  no  evidence  of  a  systemic  contract  documentation  problem  at  AFRL. 
Proposals  evaluated  under  broad  agency  announcements  may  or  may  not  be  based  on  a 
common  work  statement.  To  the  extent  they  are  not  based  on  a  common  work 
statement,  the  proposals  must  be  evaluated  based  on  varying  technical  or  scientific 
approaches. 


Recommendation,  Management  Comments,  and  Audit  Response 


Revised  and  Redirected  Recommendation.  As  a  result  of  Assistant  Secretary  of  the 
Air  Force  (Acquisition)  comments,  we  revised  draft  report  Recommendation  B.  in  the 
final  report  and  redirected  the  recommendation  to  the  Assistant  Secretary. 

B.  We  recommend  that  the  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
revise  Air  Force  Instruction  63-101,  “Acquisition  and  Sustainment  Lifecycle 
Management,”  to  require  that  price  negotiation  memorandums  explain  the 
rationale  for  selecting  the  winning  proposal  and  business  judgments  and  trade¬ 
offs  made  in  connection  with  the  selection  in  accordance  with  Federal  Acquisition 
Regulation  15.308  and  4.803  requirements. 

Air  Force  Comments  on  the  Recommendation.  The  Assistant  Secretary  non- 
concurred  with  a  draft  report  recommendation  that  the  Commander,  AFRL  revise  the 
MOASP  guidance  and  Broad  Agency  Announcement  Industry  Guide  to  require  that 
PNMs  should  explain  the  rationale  for  selecting  the  winning  proposal  and  business 
judgments  and  trade-offs  made  in  connection  with  the  selection  in  accordance  with 
FAR  15.308  and  4.803  requirements.  The  Assistant  Secretary  stated  that  the  MOASP 
will  be  incorporated  into  a  revised  version  of  Air  Force  Instruction  63-101, 
“Acquisition  and  Sustainment  Lifecycle  Management,”  by  December  2007.  The 
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Assistant  Secretary  also  noted  that  the  AFRL  Broad  Agency  Announcement  Industry 
Guide  does  not  direct  or  place  contracting  requirements  on  Government  personnel.  In 
addition,  the  Assistant  Secretary  stated  that  Air  Force  major  commands  are 
discouraged  from  further  supplementing  DoD  and  Air  Force  FAR  Supplements  as 
those  documents  adequately  cover  the  source  selection  process  to  include 
documentation. 

Audit  Response.  We  request  further  comments  from  the  Assistant  Secretary  of  the 
Air  Force  (Acquisition)  on  the  revised  Recommendation  B.  by  October  29,  2007. 
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Appendix  A.  Scope  and  Methodology 


We  performed  this  audit  from  June  2006  to  July  2007  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that  we  plan  and 
perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  the  audit  objectives. 

We  reviewed  20  AFRL  cost-type  research  and  development  contracts  (see  Appendix  B 
for  a  listing  of  the  20  contracts).  The  20  contracts,  valued  in  excess  of  $828  million, 
were  judgmentally  selected  from  120  FY  2005  contracts,  each  exceeding  $5  million, 
awarded  by  AFRL  technology  directorates  using  broad  agency  announcement 
solicitation  methods. 

We  visited  the  two  AFRL  sites  (Wright-Patterson  Air  Force  Base,  Ohio;  and  Kirtland 
Air  Force  Base,  New  Mexico)  that  included  72  percent  (86  of  the  120)  of  the  FY  2005 
AFRL  contract  actions  exceeding  more  than  $5  million  each.  We  interviewed  contract 
officials  at  Wright-Patterson  and  Kirtland  to  gain  a  better  understanding  of  the  major 
events  related  to  the  contract  award  and  contract  management  activities.  We 
interviewed  selected  contractor  and  AFRL  research  and  development  program 
officials  and  visited  selected  AFRL  laboratory  facilities  where  research  and 
development  work  is  conducted.  We  examined  the  contracting  procedures  AFRL 
officials  used  in  awarding  and  monitoring  research  and  development  contracts  for 
compliance  with  applicable  sections  of  statute,  the  FAR,  and  the  Defense  Federal 
Acquisition  Regulation  Supplement.  We  reviewed  Government  contract  file  and 
program  office  documentation  including  (but  not  limited  to)  basic  contracts,  broad 
agency  announcements,  statements  of  work,  PNMs,  technical  evaluations,  legal 
reviews,  award  fee  plans,  contract  award  announcements,  and  contract  modifications. 
We  visited  and  interviewed  management  at  AFRL  Headquarters  activities  that  oversee 
AFRL  operations.  We  also  reviewed  AFRL  internal  control  programs  as  they  related 
to  management  of  AFRL  research  and  development  contracts. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Government  Accountability  Office  High-Risk  Area.  The  Government 
Accountability  Office  has  identified  several  high-risk  areas  in  DoD.  This  report 
provides  coverage  of  the  “DoD  Contract  Management”  high-risk  area. 


Prior  Coverage 


During  the  last  5  years,  the  Government  Accountability  Office  and  the  DoD  Inspector 
General  have  issued  two  reports  discussing  research  and  development  contracting  and 
surveillance  issues.  Unrestricted  GAO  reports  can  be  accessed  over  the  Internet  at 
http://www.gao.gov.  Unrestricted  DoD  Inspector  General  reports  can  be  accessed  at 
http://www.dodig.mil/audit/reports. 
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General  Accountability  Office 

General  Accountability  Office  Report  No.  GAO-05-274,  “Opportunities  to  Improve 
Surveillance  on  DoD  Service  Contracts,”  March  17,  2005 

DoD  Inspector  General 

DoD  Inspector  General  Report  No.  D-2007-036,  “Contracting  Practices  at  the  Major 
Range  and  Test  Facilities  Base,”  December  27,  2006 
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Appendix  B.  Contract  Issues 


Contract  Number 

Contract 

Type* 

Name  of 

Contractor 

Dollar  Value 

Source 

Selection 

Negotiation 

Memo 

COTR 

Appointed 

Surveillance 

Plan 

1)  FA945 1 -05-C-0257 

CPAF 

Boeing  LTS,  Inc. 

$223,768,539 

YES 

NO 

NO 

2)  FA9453-05-C-024 1 

CPFF 

Honeywell  International 

Inc. 

26,395,461 

YES 

NO 

NO 

3)  FA945 1 -06-D-0 118 

Task  Order  0001 

CPFF 

Boeing-SVS,  Inc. 

26,381,443 

YES 

NO 

NO 

4)  FA9453-05-C-0 177 

CPAF 

Scientific  Research 

Corporation 

22,421,975 

YES 

NO 

NO 

5)  FA9453-05-D-025 1 

CPAF/FFP 

Millennium  Space 
Systems,  Inc. 

20,300,000 

YES 

NO 

NO 

6)  FA9453-05-C-0 1 65 

CPFF 

Boeing  Satellite 

Systems,  Inc. 

19,529,072 

YES 

NO 

NO 

7)  FA945 1 -04-D-03 99 

Task  Order  0002 

CPFF 

Schafer  Corporation 

17,855,489 

YES 

NO 

NO 

8)  FA9453-05-C-0220 

CPAF 

Raytheon  Company 

13,359,364 

NO 

NO 

NO 

9)  FA945 1 -05-D-0004 

Task  Order  0001 

CPFF 

Science  Applications 

International 

8,732,328 

YES 

NO 

NO 

10)  F29601-03-C-0061 

Modification  POO  14 

CPFF 

Northrop  Grumman 

Space  &  Mission 

6,025,379 

NO 

NO 

NO 

11)  FA8650-05-D-6502 

CPFF 

L-3  Communications 

Corporation 

240,900,000 

YES 

NO 

NO 

12)  F33615-03-D-2354 

Task  Order  0009 

CPFF 

Pratt  &  Whitney 

38,556,484 

YES 

NO 

NO 

13)  FA8650-04-C-7146 

CPFF 

Raytheon  Company 

26,863,120 

YES 

NO 

NO 

See  acronym  list  at  the  end  of  this  Appendix. 
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Contract  Number 

Contract 

Type 

Name  of 
Contractor 

Dollar  Value 

Source 

Selection 

Negotiation 

Memo 

COTR 

Appointed 

Surveillance 

Plan 

14)  FA8650-05-D-6633 

CPFF 

Northrop 

Grumman 

Information 

Technology,  Inc. 

24,900,000 

YES 

NO 

NO 

15)  F33615-03-D-2352  Task 

Order  0007 

CPFF 

General  Electric 

Company 

23,683,600 

YES 

NO 

NO 

16)  FA8650-04-C-7 1 08 

Modification  P00013 

CPFF 

Lockheed  Martin 

Corporation 

21,981,777 

YES 

NO 

NO 

17)  F33615-03-C-5508 

Modification  P00008 

Cost  Sharing 

Superpower,  Inc. 

10,697,440 

YES 

NO 

NO 

18)  FA8650-05-D-5052 

Cost  Sharing 

University  of 
Dayton 

20,000,000 

YES 

NO 

NO 

19)  FA8650-05-C-72 14 

Modification  P000 1 1 

CPFF 

Rockwell  Collins, 

Inc. 

19,000,000 

YES 

NO 

NO 

20)  FA8650-04-D-5233 

CPFF 

UES,  Inc. 

17,186,000 

YES 

NO 

NO 

Total 

$828,537,471 

Acronyms 

CPAF  Cost-Plus- Award-Fee 

CPFF  Cost-Plus-Fixed-Fee 

FFP  Firm-Fixed-Price 
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Appendix  C.  Report  Distribution 

Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Director,  Acquisition  Resources  and  Analysis 
Director,  Defense  Procurement  and  Acquisition  Policy 
Under  Secretary  of  Defense  (Comptroller)/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Director,  Program  Analysis  and  Evaluation 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Commander,  Air  Force  Materiel  Command 

Commander,  Air  Force  Research  Laboratory 

Air  Force  Program  Executive  Officer,  Combat  and  Mission  Support 

Auditor  General,  Department  of  the  Air  Force 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 

Senate  Committee  on  Homeland  Security  and  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 

House  Committee  on  Anned  Services 

House  Committee  on  Oversight  and  Government  Refonn 

House  Subcommittee  on  Government  Management,  Organization,  and  Procurement, 
Committee  on  Oversight  and  Government  Reform 
House  Subcommittee  on  National  Security  and  Foreign  Affairs, 

Committee  on  Oversight  and  Government  Reform 
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Department  of  the  Air  Force  Comments 


DEPARTMENT  OF  THE  AIR  FORCE 

WASHINGTON  DC 

mm 

OFFICE  OP  THE  ASSISTANT  SECRETARY 


MEMORANDUM  FOR  DEPUTY  INSPECTOR  GENERAL  FOR  AUDITING 
OFFICE  OF  THE  INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 

I 

FROM:  SAF/AQ 

SUBJECT:  DoD  Draft  Audit  Report  Contracting  Practices  at  Air  Force  Laboratory  Facilities 
(9  Ju]  07),  Project  No.  D2006-D0OOAB-02 1 7,000 

I  appreciate  the  opportunity  to  review  and  comment  on  your  draft  report  of  audit 
concerning  Contracting  Practices  at  Air  Force  Research  Laboratory  (AFRL)  facilities.  I  concur 
with  intent  on  the  findings  and  recommendations  of  the  subject  draft  report^ 

Adequate  emphasis  needs  to  be  placed  on  service  contracts  to  protect  the  interests  of  the 
Government.  The  Air  Force  has  increased  the  emphasis  on  defining  the  surveillance 
requirements,  in  the  form  of  Air  Force  Instructions  (AFI)  and  Management  and  Oversight  of 
Acquisition  Services  Programs  (MOASP)  policy,  to  ensure  appropriate  surveillance  of  all 
contracts,  including  research  and  development  contracts. 

The  AF  Program  Executive  Officer  for  Combat  and  Mission  Support  (AFPEO/CM)  is 
working  with  my  acquisition  policy  division  incorporating  the  Acquisition  (Services  Chapter 
within  die  63-series  instructions.  It  is  my  intention  to  replace  the  USAF  MOASP  policy  with 
this  new  policy  guidance.  I  expect  the  revised  AFI  63-101  to  he  published  in  December  2007. 

The  SAF/AQ  point  of  contact  for  this  audit  is  Maj  Montler,  AFPEO/CM,  at  (703)  588- 

7192, 


Assistant  Secretary  of  the  Air  Force 
—(Acquisition)-  - 

Attachments: 

Management  Comments 
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Management  Comments,  Project  No.  D2006-D0O0AB-0217.00Q 

1.  DoD  Audit  Finding  A,  Contract  Surveillance  at  Air  Force  Research  Laboratory 
Facilities:  AFRL  did  not  place  adequate  emphasis  on  monitoring  contracts  at  the  research  labs 
and  did  not  develop  QASPsfor  any  of  the  20  research  and  development  contracts  reviewed.  In 
addition,  AFRL  contracting  officers  did  not  appoint  COTRs  for  any  of  the  20  contracts.  This 
occurred  because  Air  Force  policy  for  surveillance  is  ambiguous  and  is  based  on  past  AFRL 
practices.  Although  contractors  generally  submit  project,  technical,  and  fund  status  reports,  this 
is  not  adequate  substitute  for  Air  Force  surveillance.  As  a  result,  the  AFRL  put  the  government 
at  risk  for  increased  research  and  development  costs  and  technical  risk. 

Management  Comments. 

Concur  with  Comment 


a.  The  Air  Force  has  increased  the  emphasis  on  defining  the  requirements  to  ensure  surveillance 
duties  and  responsibilities  are  adequately  performed  and  documented.  Air  Force  Instruction 
(AH)  63-124,  directs  performance-based  procedures  for  developing  requirements,  acquiring 
services,  and  managing  service  acquisitions  within  the  Air  Force,  AFI 63-124,  Paragraph  1.8 
allows  the  multifunctional  team  to  develop  an  acquisition  strategy  that  deviates  from  the 
instruction  as  long  as  it  meets  the  spirit  and  intent  of  the  requirements  of  the  AFI.  However,  the 
requirement  to  document  their  decision  to  deviate  from  the  requirements  in  the  Performance  Plan 
or  Quality  Assurance  Surveillance  Plan  must  be  documented  in  the  contract  file.  As  such,  AFRL 
used  the  AFRL  MOASP  to  ensure  surveillance  duties  and  responsibilities  were  adequately 
performed  and  documented. 

b.  The  purpose  of  the  AFRL  MOASP  is  to  implement  USAF  and  AFMC  MOASPs  for  review 
and  approval  of  all  R&D  service  acquisitions  within  AFRL,  apply  AFI  63-124  principles  to  the 
unique  R&D  services  environment,  and  establish  AFRL  R&D  services  management  and 
oversight  processes.  This  document  applies  to  all  AFRL  contract  or  task  orders  for  R&D  and 
therefore  serves  as  documentation  of  AFRL  compliance  with  AFI  63-124  and  Management  and 
Overeight  of  Acquisitions  of  Services  Processes  (MOASP)  applicable  until  NDAA  2006  change, 

c.  AFRL  manages  their  contracted  and  in-house  scientific  research  and  development  in 
accordance  with  AFPD  61-1,  Management  of  Science  and  Technology,  and  AFRLI  61-202, 
AFRL  Laboratory  Management  Review  (LMR)  Process.  The  I  JvIR  process  ensures  the 
appropriate  management  levels  have  accurate,  timely,  and  pertinent  information  upon  which  to 
make  sound  and  timely  decisions.  The  LMR  is  designed  to  rapidly  surface  potential  problems 
with  the  goal  of  maximizing  the  return  on  invested  resources.  The  LMR  establishes  and  reports 
on-eost,  schedule,  and  technical-milestones  for  R&D  effoftsTo  include  aifassessment  of  funding 
levels,  funds  status,  and  cost  performance  to  determine  if  the  objectives  of  the  contract  are  being 
met.  In  addition,  AFRL  Guidance  Memorandum  (GM  61-02-2006)  Program  Baseline 
Development  (PBD)  requires  semi-annual  baseline  reviews  on  all  R&D  contracts.  The  Program 
Baseline  Reviews  (PBR)  cover  the  program  health  and  status  regarding  cost,  schedule,  technical 
performance,  systems  engineering,  program  risk,  funding,  facilities,  manning,  contracting, 
deliverables,  and  technology  transition  plans.  The  program  managers  or  project  engineers 
review  these  PBRs  with  the  contractors  and  this  includes  the  delivery  of  reports  using  Contract 
Data  Requirement  Lists  (CDRLs).  The  LMR  and  PBR  assessments  are  used  in  lieu  of  a 
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standardized  quality  assurance  surveillance  plan  (QASP)  or  performance  plan.  Also,  the 
program  munagers/project  engineers  are  the  ad  hoc  appointed  COTRs  on  AFRL  R&D  contracts. 
Individuals  will  be  appointed  in  writing  and  the  documentation  will  be  placed  in  the  contract  file. 

2.  Da D  Audit  Finding  A  Recommendation:  We  recommend  that  the  Assistant  Secretary  of  the 
Air  Force  ( Acquisition )  require,  the.  Air  Force  Research  Laboratory  to  re\'ise  the  Management 
and  Oversight  of  Acquisition  Services  Processes  to  include  guidance  that  each  Air  Force 
Research  Laboratory  develop  a  quality  assurance  surveillance  plan  for  each  contract  to  reduce 
the  risk  of  the  government  paying  more  for  services  than  the  value  received  and  to  ensure 
surveillance  responsibilities  are  being  carried  out.  In  addition,  the  guidance  should  clearly 
define  the  roles  and  responsibilities  of  contacting  personnel  to  include  appointing  a  trained 
Contacting  Officer's  Technical  Representative  to  monitor  technical  service  performance 
contracts. 

Management  Comments. 

Concur  with  Comment 

a.  A  draft  API  63-101,  Acquisition  and  Sustainment  Lifecycle  Management,  is  being  coordinated 
within  the  Air  Staff.  Chapter  3,  Acquisition  of  Services,  will  replace  the  USAF  MOASP  policy. 
The  estimated  date  for  the  approval  of  the  revised  AH  63-101  is  December  2007. 

b.  AFRL  utilizes  unique  oversight  procedures  such  as  Program  Management  Reviews  (PMRs), 
LMRs,  Technical  Management  Reviews  (TMRs)  and  review  of  contractor  generated  data  such  os 
reports  required  by  the  CDRLs  and  other  deliverable  reports  to  ensure  quality.  Progress  toward 
objectives  is  measured  through  periodic  review  of  contractor  cost,  schedule  and  technical 
performance.  The  uncertain  nature  of  R&D  precludes  development  and  application  of  objective 
criteria  for  performance  assessment  used  with  traditional  service  contracts. 

3.  Dol)  Audit  Finding  B,  Air  Force  Research  Laboratories  Research  and  Development 
Contracts  Source  Selection  Process:  AFRL  established  adequate  internal  controls  over  the 
research  and  development  contract  award  process  and  compliance  with  applicable  sections  of 
the  FAR.  AFRL  contract  file  documents  for  18  of  20  contracts  we  reviewed  contained  an 
overview  of  the  source  selection  identifying  the  original  bidders  and  explaining  how  the  winner 
was  selected,  even  though  AFRL  did  not  have  a  contracting  policy  to  require  documentation  of 
source  selection  reasoning.  Without  source  selection  documentation,  an  audit  trail  of  the  major 
procuring  events  is  unavailable  resulting  in  loss  of  the  history  of  business  judgments  and  trade¬ 
offs  made  by  the  government,  including  benefits  associated  with  additional  cost  to  the 
acquisition. 

Management  Comments! 

Concur 

a.  I  agree  that  documentation  of  the  source  selection  process  for  Request  for  Proposal  (RFP)  in 
accordance  with  FAR  Part  15  or  documentation  of  die  peer  or  scientific  review  in  the  case  of  a 
Broad  Agency  Announcement  (BAA)  in  accordance  with  FAR  35.016,  is  required  and  must  be 
included  in  the  contract  file.  Proposals  received  as  a  result  of  the  BA  A  are  evaluated  in 
accordance  with  evaluation  criteria  specified  in  the  announcement  through  a  peer  or  scientific 
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Final  Report 
Reference 


Revised  and 
Redirected 


review  process,  Written  evaluation  reports  on  individual  proposals  are  accomplished,  but 
proposals  are  not  necessarily  evaluated  against  each  other  since  they  are  not  submitted  in 
accordance  with  a  common  work  statement. 

b.  Due  to  existing  AFFARS  and  AFMC  guidance,  AFRL  does  not  require  additional  policy 
instructions  for  selection  documentation  and  price  reasonableness  determinations. 

c,  I  concur  that  contract  F29601-03-C-0061  did  not  contain  adequate  documentation  of  the 
selection  decision  in  either  the  technical  evaluation  or  negotiation  memorandum;  however,  this  is 
an  isolated  incident,  and  there  is  no  systemic  problem  across  AFRL.  This  contract  is  completed; 
however,  the  need  for  inclusion  of  documentation  on  the  acquisition  process  will  be  included  in 
training  for  all  contracting  officers/contract  negotiators. 

4.  Do!)  Audit  Finding  B  Recommendation;  We  recommend  that  the  Commander,  Air  Force 
Research  Laboratory  revise  the  Management  and  Oversight  of  Acquisition  of  Services  Processes 
guidance  and  Broad  Agency  Announcement  Industry  Guide  to  require  that  negotiation 
memorandums  should  explain  the  rationale  for  selecting  the  winning  proposal  and  business 
judgments  and  trade-offs  nude  in  connection  with  the  selection  in  accordance  with  Federal 
Acquisition  Regulation  15.308  and  4.803  requirements. 

Management  Comments. 

Non-concur 

a.  The  Management  and  Oversight  of  Acquisition  Services  Process  (MOASP)  will  be 
incorporated  in  the  next  version  of  API  63-101  which  is  currently  in  coordination.  Once  the  AF1 
is  published,  the  USAF  MOASP  will  no  longer  exist. 

b.  In  addition,  the  “Broad  Agency  Announcement  Industry  Guide”  is  a  guide  and  is  offered  for 
information  to  contractors  and,  as  a  guide,  does  not  direct  or  place  requirements  on  government 
contracting  personnel.  There  are  no  requirements  to  provide  additional  guidance  at  the  AFRL 
level.  The  MAJCOMs  are  discouraged  from  supplementing  AFFARS  since  the  FAR,  DFARS, 
AFFARS,  and  AFMCFARS  adequately  cover  the  source  selection  process,  to  include 
documentation. 
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